机器学习（ML）涵盖的实验必须考虑评估模型性能的两个重要方面：数据集和算法。需要强大的基准来评估最佳分类器。为此，可以采用公共存储库中提供的金标准基准。但是，常常不考虑在评估时考虑数据集的复杂性。这项工作提出了一种基于物品响应理论（IRT）和GLICKO-2的组合的新评估方法，该方法通常采用了评估参与者的强度（例如，国际象棋）。对于基准测试中的每个数据集，IRT用于估计分类器的能力，良好的分类器对最困难的测试实例具有良好的预测。然后为每对分类器运行锦标赛，以便GLICKO-2更新每个分类器等额定值，评级偏差和波动等性能信息。在此进行了一个案例研究，该研究通过了OpenML-CC18基准作为数据集的集合和各种分类算法的池进行评估。并非所有数据集都被观察到对评估算法非常有用，其中只有10％被认为是非常困难的。此外，验证了仅包含50％的OpenML-CC18的50％的子集的存在，其同样有用于算法评估。关于算法，本文提出的方法将随机林识别为具有最佳天生能力的算法。

基于可解释的人工智能 - XAI的策略在计算中促进了更好地了解黑匣子模型的预测。今天使用的大多数XAI措施解释了这些类型的模型，生成了旨在解释模型的属性排名，即模型的属性重要性分析。 Xai措施没有共识，Xai措施产生了整体解释性等级。因此，出现了几种工具的提案（CIU，DALEX，ELI5，LOFO，SHAP和SPRATER）。本文介绍了一种能够产生全球解释性等级的可解释的AI技术的实验基准。寻求回答“诸如”不同措施产生的解释，相同或不同？“和“数据复杂性如何沿模型解释性发挥作用？” 82个计算模型的构造结果和592的速度排列在解释性问题的另一面上的一些光线：数据集复杂性！

The use of reinforcement learning has proven to be very promising for solving complex activities without human supervision during their learning process. However, their successful applications are predominantly focused on fictional and entertainment problems - such as games. Based on the above, this work aims to shed light on the application of reinforcement learning to solve this relevant real-world problem, the genome assembly. By expanding the only approach found in the literature that addresses this problem, we carefully explored the aspects of intelligent agent learning, performed by the Q-learning algorithm, to understand its suitability to be applied in scenarios whose characteristics are more similar to those faced by real genome projects. The improvements proposed here include changing the previously proposed reward system and including state space exploration optimization strategies based on dynamic pruning and mutual collaboration with evolutionary computing. These investigations were tried on 23 new environments with larger inputs than those used previously. All these environments are freely available on the internet for the evolution of this research by the scientific community. The results suggest consistent performance progress using the proposed improvements, however, they also demonstrate the limitations of them, especially related to the high dimensionality of state and action spaces. We also present, later, the paths that can be traced to tackle genome assembly efficiently in real scenarios considering recent, successfully reinforcement learning applications - including deep reinforcement learning - from other domains dealing with high-dimensional inputs.

We study inductive matrix completion (matrix completion with side information) under an i.i.d. subgaussian noise assumption at a low noise regime, with uniform sampling of the entries. We obtain for the first time generalization bounds with the following three properties: (1) they scale like the standard deviation of the noise and in particular approach zero in the exact recovery case; (2) even in the presence of noise, they converge to zero when the sample size approaches infinity; and (3) for a fixed dimension of the side information, they only have a logarithmic dependence on the size of the matrix. Differently from many works in approximate recovery, we present results both for bounded Lipschitz losses and for the absolute loss, with the latter relying on Talagrand-type inequalities. The proofs create a bridge between two approaches to the theoretical analysis of matrix completion, since they consist in a combination of techniques from both the exact recovery literature and the approximate recovery literature.

This paper presents a corpus annotated for the task of direct-speech extraction in Croatian. The paper focuses on the annotation of the quotation, co-reference resolution, and sentiment annotation in SETimes news corpus in Croatian and on the analysis of its language-specific differences compared to English. From this, a list of the phenomena that require special attention when performing these annotations is derived. The generated corpus with quotation features annotations can be used for multiple tasks in the field of Natural Language Processing.

With the ever-growing popularity of the field of NLP, the demand for datasets in low resourced-languages follows suit. Following a previously established framework, in this paper, we present the UNER dataset, a multilingual and hierarchical parallel corpus annotated for named-entities. We describe in detail the developed procedure necessary to create this type of dataset in any language available on Wikipedia with DBpedia information. The three-step procedure extracts entities from Wikipedia articles, links them to DBpedia, and maps the DBpedia sets of classes to the UNER labels. This is followed by a post-processing procedure that significantly increases the number of identified entities in the final results. The paper concludes with a statistical and qualitative analysis of the resulting dataset.

This article presents the application of the Universal Named Entity framework to generate automatically annotated corpora. By using a workflow that extracts Wikipedia data and meta-data and DBpedia information, we generated an English dataset which is described and evaluated. Furthermore, we conducted a set of experiments to improve the annotations in terms of precision, recall, and F1-measure. The final dataset is available and the established workflow can be applied to any language with existing Wikipedia and DBpedia. As part of future research, we intend to continue improving the annotation process and extend it to other languages.

Evaluating new techniques on realistic datasets plays a crucial role in the development of ML research and its broader adoption by practitioners. In recent years, there has been a significant increase of publicly available unstructured data resources for computer vision and NLP tasks. However, tabular data -- which is prevalent in many high-stakes domains -- has been lagging behind. To bridge this gap, we present Bank Account Fraud (BAF), the first publicly available privacy-preserving, large-scale, realistic suite of tabular datasets. The suite was generated by applying state-of-the-art tabular data generation techniques on an anonymized,real-world bank account opening fraud detection dataset. This setting carries a set of challenges that are commonplace in real-world applications, including temporal dynamics and significant class imbalance. Additionally, to allow practitioners to stress test both performance and fairness of ML methods, each dataset variant of BAF contains specific types of data bias. With this resource, we aim to provide the research community with a more realistic, complete, and robust test bed to evaluate novel and existing methods.

对制造工艺的机器化的需求很大，因此单调劳动。一些需要特定技能的制造任务（焊接，绘画等）缺乏工人。机器人已在这些任务中使用，但是它们的灵活性受到限制，因为它们仍然很难通过非专家编程/重新编程，从而使它们无法访问大多数公司。机器人离线编程（OLP）是可靠的。但是，直接来自CAD/CAM的生成路径不包括代表人类技能的相关参数，例如机器人最终效应器的方向和速度。本文提出了一个直观的机器人编程系统，以捕捉人类制造技能并将其转变为机器人程序。使用连接到工作工具的磁跟踪系统记录人类熟练工人的演示。收集的数据包括工作路径的方向和速度。位置数据是从CAD/CAM中提取的，因为磁跟踪器捕获时的误差很明显。路径姿势在笛卡尔空间中转换，并在模拟环境中进行验证。生成机器人程序并将其转移到真正的机器人。关于玻璃粘合剂应用过程的实验证明了拟议框架捕获人类技能并将其转移到机器人方面的使用和有效性的直觉。

我们介绍了IST和Unmabel对WMT 2022关于质量估计（QE）的共享任务的共同贡献。我们的团队参与了所有三个子任务：（i）句子和单词级质量预测；（ii）可解释的量化宽松；（iii）关键错误检测。对于所有任务，我们在彗星框架之上构建，将其与OpenKIWI的预测估计架构连接，并为其配备单词级序列标记器和解释提取器。我们的结果表明，在预处理过程中合并参考可以改善下游任务上多种语言对的性能，并且通过句子和单词级别的目标共同培训可以进一步提高。此外，将注意力和梯度信息结合在一起被证明是提取句子级量化量化宽松模型的良好解释的首要策略。总体而言，我们的意见书在几乎所有语言对的所有三个任务中都取得了最佳的结果。